phmm: stemming on persian texts using statistical stemmer based on hidden markov modelphmm: stemming on persian texts using statistical stemmer based on hidden markov model

stemming is the process of finding the main morpheme of a word andit is used in natural language processing, text mining and informationretrieval systems. a stemmer extracts the stem of the words. we can classifypersian stemmers in to three main classes: structural stemmers, dictionarybased stemmers and  statistical stemmers.the precision of structural stemmers is low and the expenses of dictionary basedstemmers is high, so the main goal of this research is to design and implementa statistical stemmer based on hidden markov model  with high precision which can reduce the sizeof indexed file and  increase the speedof information retrieval systems. our proposed stemmer, finds the prefixes and suffixes of a word and removethem, so the rest of the word is the stem. but there are some exceptions inpersian words which lead to stem those words by mistakes. so we collect a dictionaryof  persian stemmers. our proposed  stemmers, search a word  in the dictionary, if it is not there , itfinds the stem of it by hmm based stemmer. this stemmer is tested in bijankhancorpus and hamshahri test collection. the results show increment in meanaverage precision and recall. the speed of the information retrieval system isincreased and the size of  indexed filesis decreased by the algorithm.